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ABSTRACT 


Berthing of the spacecraft with non cooperative targets are gain- 
ing interest in recent days. Space robotics is considered one of 
the most promising approaches for missions such as docking, re- 
fuelling, repairing, upgrading, transporting, rescuing, and orbital 
debris removal. Many enabling techniques have been developed in 
the past two decades and several technology demonstration mis- 
sions have been completed. A number of manned on-orbit servicing 
missions were successfully accomplished but unmanned, fully au- 
tonomous, non-co-operative servicing missions have not been done 
yet. With the advent of advanced image processing algorithm and 
its hardware implementation made it possible to perform robotic 
arm actuations in space for berthing with on orbit satellite. Usually 
the robotic arm will be equipped by a small camera which will track 
the target. Here the usage of colour marker is useful as it can be 
detected and tracked faster with less amount of processing. One of 
the greatest challenges is to ensure the servicing spacecraft safely 
and reliably docks with the target spacecraft or capture the target 
to stabilize it for subsequent berthing. This is especially important 
if the target has an unknown motion and kinematics! dynamics, 
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1 INTRODUCTION 

Space Robotics deals with the development of general purpose 
machines that are capable of surviving the rigors of the space envi- 
ronment and perform exploration, assembly, construction, mainte- 
nance, servicing or other tasks. Space Robots are generally designed 
to do multiple tasks, like payload deployment, retrieval, inspec- 
tions and planetary exploration including unanticipated operations 
[1] Space robotics has shown immense potential in the recent years 
due to comparatively lower cost and lower risk factors for manned 
missions. In addition, it has supported several on-orbit operations 
to astronauts on-board, thus emphasizing its importance. One such 
important application of space roboties is docking. Docking and 
undocking of spacecraft are required for replenishment of fuel, pro- 
pellant, power, food or maintenance for space systems. Presently, 
two types of docking concepts are in practice. In the first type, the 
relative velocities between the chaser and target spacecraft are used 
to activate the docking mechanism (Figure 1) and it is known as “Im- 
pact Docking”. Typically, rigid spacecraft which are axis-symmetric 
have employed this concept. The second type of docking is the 
low impact docking concept. This concept induces low docking 
disturbance at the time of contact and the risk of damage to the 
spacecraft due to collision is minimal (2, 3]. Berthing falls in the 
category of low impact docking concepts. The ‘Berthing’ scheme 
[4,9] refers to the use of an intermediate system, typically a robotic 
arm to capture the target vehicle, Further, the robotic arm performs 
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Figure 1: Target and Chaser spacecraft with manipulator 


the maneuvers required to position the vehicles for the desired 
interface and subsequently docks the target Spacecraft to chaser 
Spacecraft by means of motor driven latches provided on the chaser 
spacecraft. The Space Shuttle manipulator/robotic arm is a classic 
example for berthing. In this approach, the risk due to collision 
is minimized. A dexterous robotic arm is useful for controlling a 
non-cooperative target with greater accuracy and alignment [5, 6]. 
ETS-7-Orbital [7] Rendezvous & Robotic Mission (JAXA), Orbital 
Express, Dexterous Robotic arm (ESA), JEM Remote manipulator 
system (JAXA), European Robotic arm (ERA), SSRMS (Canada Arm- 
2) are examples of Berthing scheme for low impact docking. In this 
paper, then overall processing pipeline is explained Initially vision 
based method is used to detect the marker and 6DOF is found out.To 
motivate and facilitate such research and development, this paper 
provides a literature review related to the kinematics, dynamics, 
control and verification of space robotic systems, the requirement 
and best positions to mount the marker, image processing pipeline 
for vision based target marker acquisition to compute their co- 
ordinates, 6DOF(degree of freedom) computation, motor selection, 
design, analysis and test envisaged for the development robotic 
arm and robotic arm gripper design and its actuation is presented. 


2 LITERATURE SURVEY 


In recent times, due to advancement of vision based system avail- 
ability with good processing power, berthing and on-orbit servic~ 
ing (OOS) of the spacecraft has gain interest over the researchers. 
[11, 13, 14]These research works were motivated by several national 
and international missions not only for repairing, rescuing, and 
refuelling failed satellites, but also for removal of defunct satellites 
or space debris [12, 17]. A space robotic system (also referred to 
as space manipulator or space robot) for an OOS mission typically 
consists of three major components: the base spacecraft or servic~ 
ing satellite, an n degree-of-freedom (n-DOF) robot manipulator 
attached to the servicing satellite, and the target spacecraft to be 
serviced. A spacecraft-manipulator servicing vehicle is sometimes 
termed the servicing system. The capturing process includes a se- 
ries of operations. After having completed the far and close-range 
rendezvous manoeuvres [13] with the target satellite, the servicing 
spacecraft remains at a safe, station-keeping, distance from the 
tumbling target satellite. Then, the capture operation mode starts, 
which may be divided into four phases. The first phase corresponds 
to the observing and planning phase for acquiring motion and phys- 
ical properties information about the target satellite, to plan how 
the robot manipulator should grasp the target. In this stage vision 
based algorithms are used to identify the target spacecraft and find 
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its location and orientation (6 DOF). In the current paper color 
based tracking [15] is used. After the colour target is located then 
4 numerical iterative method and TRIAD [16] based algorithms 
are used to precisely compute 6DOF. This information is given to 
Kinematics to compute each joint angle with the help of inverse 
Kinematics, In the second phase robotic manipulator is controlled 
based on inverse kinematics information available to move toward 
the planned grasping location, such that the robotic arm is ready 
to capture the target. The third phase consists in the actual capture 
(physical interception) phase in which the manipulator physically 
grasps the capturing device of the target satellite. The fourth phase 
is the post-capture phase, where the captured target is stabilized 
along with the servicing system. This paper [10] provides a survey 
of the literature on different topics related to space robotic systems 
for manned and unmanned on-orbit servicing mis 


ions. 


3 BASIC BLOCK DIAGRAM OF THE SYSTEM 


‘The system consists of a USB camera mounted on a robotic arm, 
control system electronics and a robotic arm with gripper and the 
corresponding drive electronics. The camera will see objects and 
will find out the colour marker location in the image frame. This 
data will be used to compute 6DOF information (3 translation and 
3 rotation angle) . This data will be used with inverse kinematics 
to compute the joint angle of the robotic arm. The robotic arm 
will have rotation provision and the arm will be stopped based 
on distances which will be available from translation, Other two 
axis translation will be used to align the entire spacecraft so that 
robotic manipulator can grip and pull the other spacecraft. The 
control electronics will decide the required movement to reach the 
target depending on joint angle. The image processing pipeline is 
explained in Figure 2 This process will be repeated till the robotic 
arm reached to the destination and able to grip the servicing satellite. 
The details step of the above block diagram is explained below. 


3.1 Image processing pipeline 

For a robotic arm to reach its needs very accurate and precise track- 
ing. It should be fast, reliable and robust. For this purpose the colour 
marker tracking is used. Colour detection can be done much fast 
and reliable way compared to other feature tracking method. The 
input colour image has 3 component ,ie. Red,Green,Blue. Initially 
it is converted to HSV(Hue, Saturation, Value) colour space Figure 
3. Here Hue is mainly used for colour thresholding. In the real 
image, other than the actual circular marker, it is possible some 
other small blobs of that particular hue may get detected. Those 
small blobs needs to be discarded for computation. Hence image 
clustering is used and they are arranged in ascending order of their 
size. The largest and 2°4 largest cluster (for red marker) and largest 
cluster for the blue marker will be selected for further computation, 
To identify the location of the marker centroid of the cluster is com- 
puted. So three centroid (x1, y1), (22, ye), (x3, ys) is available after 
image processing algorithm. This centroid location is used for (DOF 
calculation using numerical techniques as explained subsequently 
(Figure 4) 
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Figure 2: Robotic arm actuating pipeline 


Figure 3: HSV Color space Source https://linzichun.com/ 
posts/ 


3.2 6 DOF computation using TRIAD 

To compute 6DOF , three color markers are mounted on a struc- 
ture where two are in plane(Img2,Img3) and one are out of plane 
(img1)(Figure 4), This out of plane marker will enhance the rotation. 
accuracy. As explained earlier,((x1, y1)is the centroid location of the 
out of plane marker in the image and (x2, y2)& (1s, ys) are the cen- 
troid for Img2 and Img3 marker in the captured image. So initially 
for each such centroid, azimuth and elevations are computed and 
they are (1, fi), (a2, bz), (as, Bs)respectively. This azimuth and el- 
evation will be used to generate unit vector (ty, u2,us) for those 
three points. Then by finding dot product between two vectors, 
angles are calculated as follows 


Liz = (unt a) 
Las = (uz.tts 2) 
Lis = (urus @) 


From the initial position of the marker location, the vector po- 
sition is known in the target frame. k12, kzs.kisare the distance 
between marker 1&2, 2&3,183 respectively. Now a vector Riyscan 
be defined where R(1),R(2) & R(3) are the distances of color marker 
point from camera optical centre. Initially they are assumed same 
number with initial value. Now for each point three equations are 
formed and written in matrix form as below. 

R(1)? + R(2)* = 2R (1) R(2) Liz — key 
R(2)? + R(3)? — 2B (2) R (3) Las — key (4) 
R(1)? + R(3)? = 2R (1) R (3) Lis — ky 


Now newton Raphson method is used for non linear opti- 
mization. The updated value of Ry will provide range infor- 
mation. Then azimuth and elevation will be used to compute 
XYZ of each point. Now for each three point vectors are caleu- 
lated in chaser Imgl, Img2r, Ing3,_ is the position vector of 
Img, Img2, Img3_ color marker respectively. The vectors in the 
chaser frame will be 


Ride = R(1) ay ~ R (2) ay & RI = R(1) my — R(3) us (3) 
‘The vectors in the target frame will be 
Ri2y = Imgly — Inigdy & R13, = Imghy — Img’, 6) 


‘The chaser frame vectors are normalized and will be denoted 
as (W12¢, u23¢) and target frame vectors are normalized and de- 
noted by (0121, 023,), Now using the above vectors three triads 
are computed for chaser frame (t1p, 2p, 34) and target frame 
(t14,12;, 34). From these TRIAD rotation matrix can be computed 
and angles are extracted considering Euler 1-2-3 rotation, 


3.3 Lab Calibration and stability: 


‘The camera needs accurate lab calibration set up(Figure 5) to ob- 
tain a very high accuracy output. The camera is mounted on a 
rotation table to ensure zero rotation, zero translation. Then the 
image is captured (Figure 6) and camera optical centre is deter- 
mined. Similarly, computation of focal length is also done based 
on FOV. Here the extrema two position images was captured by 
rotating the turn table and thus FOV is calculated. Using that focal 
length is computed. This information is used to determine 6 DOF 
information. A linear translator is used to verify the correctness 
of the translation movement. The 30 stability plot was obtained 
for a particular position Figure 7 shown stability along x, Figure 8 
shows stability along y, Figure 9 stability along z.Camera was put 
stationary and all 3 axis translation are plotted with X axis as No of 
Sample and Y axis as translation. It is observed only along X axis ( 
along the sensor optical axis) accuracy ~2mm and other two axis it 
is <0.5mm which indicates the detection method is robust and can 
provide accurate translation information. 


4 SYSTEM DESCRIPTION 

Chaser and Target satellites are two small satellites of size 0.5m 
x0.5m x 0.5m and mass of about 100kg each [8].The manipulator 
arm with 6 DOF(Figure 1) is mounted with the base of the shoulder 
joint being fixed to the Chaser Spacecraft. 
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Figure 4: 6DOF calculation using camera 


Figure 


: Images captured in Lab with color marker detected 


‘The arm is consists of two links, Arm 1 & Arm 2 (Figure 10). The 
shoulder joint has two revolute joints, Joint 1 & Joint 2 which pro- 
vides rotation about roll & pitch axis. The two links are connected 
by the elbow joint, Joint 3. At the end of Arm 2 is the wrist mecha 
nism which provides for rotation about three perpendicular axes 
(Joints 4, 5 & 6) and constitutes a two finger gripper at its end which 
is meant for capturing the Target Spacecraft. The gripper consists of 


‘a ball screw and four bar mechanism for operating the two fingers 
which captures the grappling rod on the target Spacecraft. 


5 ANALYSIS AND MODELLING 
A berthing analysis has been carried out using ADAMS 2012 soft- 
ware for the evaluation of the motor torque requirements for the 
robotic arm, Berthing constitutes of two basic steps: 

Capture: The maneuvering of the robotic arm to capture the 
target spacecraft. 

Retraction: The maneuvering of the robotic arm to bring the 
grappled target within close proximity of docking mechanism, 

‘The second step is more critical for the evaluation of the motor 
torque requirements as the inertia attached to the robotic arm is 
higher. The robotic arm is modeled in ADAMS with 6 rotational 
DOF as shown . All the six joints in the robotic arm are modeled 
with revolute joints. The motors of the robotic arm are considered 
to operate with a constant angular velocity of 1deg/sec. The robotic 
joint is mounted over the chaser spacecraft. The target spacecraft is 
modeled as a freely suspended body in space. During capture and 
retraction the moments at the joints are evaluated. The maximum. 
initial relative velocity and angular velocity imparted to the target 
are Imm/sec and 0.01deg/sec at the start of the movement of the 
robotic arm from the chaser. It is assumed that during the capture 
the thrusters are inactive thus rendering the chaser unstable while 
during retraction chaser spacecraft is stabilized by the thrusters. 
Hence during retraction analysis all the 6 DOF of the chaser are 
constrained [9]. The analysis is also carried out for the case where 
the thrusters are inactive during retraction. Inactivated thrusters 
can be simulated by releasing all the 6 DoF of the chaser space- 
craft.All the three cases the maximum torque required by different 
‘motors are tabulated in the Table 1. 


5.1 Case1 


For the first case the target spacecraft translates relative to the 
chaser spacecraft about the three translational axes with a magni- 
tude of Imm/sec each. The chaser spacecraft is constrained about 
all 6 DOF during retraction. 
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Figure 9: 


5.2 Case 2 

For the second case in addition to imparting the relative transla- 
tional velocities as in case-1 relative angular velocity of 0.01 deg/see 
is also imparted to the target about the three axes with respect to 
the chaser. The chaser spacecraft is constraint about all 6 DOF 
during retraction. 


(mm) 


150 200 250 300 


stability plot along X axis 


5.3 Case 3 

In first and second case the chaser spacecraft is stabilized by 
thrusters while in the third case the thrusters are inactive, Hence, 
chaser is in a free-free condition. The initial conditions are same as 
that of Case-2, The torque requirement for the motor gets reduced 
considerably. 
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Spacecraft 


Figure 10: Configuration of the robotic arm. 


‘Table 1: Study of torque requirement for various configurations 


Joint ID ‘Torque Requirement(Nm) Case Torque Requirement(Nm) Case2 Torque Requirement(Nm) 
Case3 
1 12 25 04 
2 2 50 0.45 
3 26 35 0.85 
4 19 19 0.85 
5 18 135 0.39 
6 04 02 03 


5.4 Observations 

Cases (1&2) where the chaser is constrained the reaction moments 
generated by the thrusters get transferred to the joints thus inducing 
greater load on the motors. While in case-3 all DOF of the chaser are 
released. Hence the chaser is in a free-free condition and the motor 
joints thereby see reduced torque requirements, During the analysis 
it is assumed that the motors operate at 1deg/sec. On reducing the 
operating speed, the torque requirement may further drop down. 
The torque requirements are maximum for Case-2 which wei 
considered as design input for motor selection To maintain motor 
at nominal operating temperatures foil heaters are bonded on the 
motor housing. The metallic components are anodized to keep 
temperatures within nominal ranges. 


6 ROBOTICS DESIGN OF ARM 

Design trade off and options study was performed to arrive at the fi- 
nal configuration of the manipulator arm (18]:The manipulator arm 
consists of three important subassemblies: a) Shoulder and elbow 
joints b) Wrist mechanism e) End effector/Grippet Mechanism 


6.1 Shoulder and elbow joints. 

The configuration of both shoulder and elbow joints are kept similar 
based on their capability to drive higher inertia as compared to the 
wrist joints, Each joint is driven using a 6 Nm capacity BLDC motor 
with planetary gear head and encoders for position feedback. The 
motor shaft is supported on a pair of radial ball bearings to resist 
the moment loads coming due to the tip mass in fully stretched 


configuration at a distance of 1 m from the manipulator base. The 
bearings are kept in a preloaded condition by suitably tolerancing 
the mating components such as clamp, rotor bracket and spacer. 
The inner race of the bearing is mounted on the stator and the 
outer race is connected to the rotor. The rotor is coupled to the 
shaft using a single D joint to prevent any relative motion between 
the two parts. 


6.2 Wrist Mechanism. 

‘Minimum 6 Degrees of freedom (DOF) are required to capture target 
spacecraft in space by a free flying manipulator. Therefore Wrist 
mechanism consists of three non-intersecting joints to facilitate 
fine adjustment in all the three rotational axes required during 
operation. Since the moment load due to inertia is less at the wrist; 
smaller 1.5 Nin BLDC motor with planetary gear head and encoders 
is used to drive the joints. In wrist joint motor, shaft is coupled 
to an output shaft using a flexible metallic coupling to cater for 
any mismatch due to fabrication errors, Further the output shaft is 
supported on a pair of radial ball bearings similar to shoulder joints. 
Only difference is that the inner race of the bearing is connected to 
the rotating shaft and the outer race is connected to the stationary 
housing, At the end of the output shaft is a double D interface to 
connect the housing of the other wrist joint. Gripper mechanism is 
connected at the end of the third wrist joint. 
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Figure 11: GUI of the software 


6.3 Gripper Mechanism. 

The gripper is actuated using a ball screw mechanism driven by 
a brush DC motor. As the motor rotates the screw & the nut pair 
can move linearly in either direction thus actuating the four bar 
mechanism which thereby either closes or opens the two fingers. 
The friction due to relative motion between the links is reduced by 
mounting a du bush at the joints. To provide a positive lock to the 
gripper a rest pad is provided which act as a third gripping point 
ensuring that the target is sufficiently inside the triangle formed by 
the rest pad and the two fingers. Gripper pads are connected at the 
tip of each finger which serves as a housing for sensors for gripper 
force measurement and acts as a non-contact type target locator. 


7 TARGET RECOGNITION AND CAPTURING 
SOFTWARE 


‘Target Recognition and Automated Capturing Software (TRACS) is 
developed to operate the robotic arm. Figure 11 shows the GUI of 
the developed software. The designed software has 5 modules. 


1, The first module uses ETS to define a robotic arm, Any gen- 
eralized robotic arm can be defined using this module with 
user defined degree of freedom. Once the robotic arm has 
been defined the software can be used to view the simplified 
model of the robotic arm as shown in Figure 12 

2. The second module of the software does inverse kinematics 
for the defined robotic arm. It evaluates the motor rotations 
required for the robotic arm to transverse from the initial 
configuration to the required pose. 

3. The third module of the software interfaces with the camera 
to receive the target position in real time. 


4. The fourth module is interfaced with the drive electronics 
of the motor to arbitrarily command motor rotations. 

5. The fifth module of the software integrates the first four 
modules to track the target automatically in real time and 
then capture the interface ring. 

6. The software interfaces with the camera based pose estima- 
tion program to receive the pose of the camera mounted on 
the robotic arm w.rt target. The evaluated transformation 
matrix can be inverted to get the pose of the target w.rt cam- 
era, The encoders present in the motors provide individual 
rotated angles of each motors present in the robotic arm, 
Using the encoder data the pose of the robotic arm can be 
evaluated. The software calculates the following transfor- 
‘mation matrix in real time during the tracking of the target. 


capturing Pos 
saeiee "4 P°*7 = axa transformation matrix of the capturing position 
on the ring w.rt target (Defined by geometry) 


(arget,T = 4x4 transformation matrix of the target w.r.t camera 


(Evaluated using the camera feedback) 


carrera yoy! = 4x4 transformation matrix of the camerat wr. End- 


effector (defined by geometry) 


naeffectory = 4x4 transformation matrix of the End-effector 


wert base (evaluated using the encoder feedback) 


capturing pos, _ capturing Pos target camerat end ef fector 

sane PT = ahget Teal Send cf fector™ Nbane T 
Product of the transformation matrix of the robotic arm and 

the target w.rt the camera gives the overall transformation matrix 

of the target wart base of the arm. The software takes feedback 

from the encoder data and from the camera and in real-time derives 

the current pose of the robotic arm as well as the transformation 
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Figure 13: Robotic arm manipulator with all joints 


matrix for the gripper to reach the target-The software uses inverse 
Kinematics to evaluate the additional rotations required by the 
motors reach the target. The software commands the motor to 
operate for the required angles. These steps are executed in a loop 
as long as the distance between the target and the gripper is within 
the set tolerance limit or the proximity sensor in the gripper gets 
actuated. Once the tracking of target is over, time based series of 
commands are run to operate the gripper for capturing the target. 
On successful capture of the target the robotic arm is brought to its 
rest pose along with the target. 


8 ROBOTIC ARM AVIONICS 


Drive Electronics is built to cater for driving 8 Brushed DC Motors 
(BDC) simultaneously in close loop. Major components of avionics 
include Teensy 3.5 board (32bit-120MHz Arm cortex-M4 floating 
point microcontroller), Dual-H-bridges (L298) and other peripheral 
circuitries. Out of eight motors, six are for arm joints, two are 
for gripper. Six arm motors include two at shoulder (246:1 gear 
train), one at elbow (158:1 gear train), and three at wrist (158:1 gear 
train). Figure 13 shows robotic arm configuration with all motors 
nomenclature. Each ARM-BDC motor is housed with incremental 
magnetic encoder to provide real time shaft position. These are 9 
bit encoders. Encoder signals (A & B) are captured and processed 
by microcontroller. By processing both rising and falling edge of 
encoder signals, the positional resolution is further enhanced by 4 
times, For a full motor range of 360°, encoder provides resolution 
as 0.176" 

360° 


oe 176° 
(1) 


Encoder Resolution = 


8.1 Drive Electronics Design & Realization 

Figure 14 shows basic block diagram and interfaces of drive elec- 
tronics.All motors are powered with single 42V power supply. Pulse- 
width-modulation (PWM) based variable speed controller is imple- 
‘mented to run each motor at different voltage and speed. All six 
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Figure 15: Control loop diagram 


arm motors are controlled using PD controller (position and rate 
control) with encoder in close loop based on commanded angle. 
Figure 15 shows control loop block diagram.Innovative mixed brak- 
ing mechanism (mixture of slow and fast braking) for BDC motor 
is implemented in this design which avoids free running of motor 
when supply is cut-off. Each motor module has limit switches at ei- 
ther end, These limit switches get activated once motor shaft comes 
in contact with them. When these limit switches are activated, re- 
spective motor rotation is stopped. Proximity sensor interface is 
also included in this hardware. Proximity sensor is triggered when 
robotic arm reaches near the target, thereby switching off all the 
six arm motors.Gripper motors comprise of one horizontal actuator 
and one vertical actuator. These gripper motors are controlled based 
on commanded run-time & individual limit switches. 


8.2 Implementation: 

Drive Software is implemented using Embedded C/C++ language 
using Arduino IDE. Software has provision of commanding absolute 
angles (+360" range), with added features such as presetting of 
present angle, pulse width selection (Resolution: 1us, up to seconds), 
‘enabling or disabling limit switches and proximity sensors logics 
etc, Modes of operation include idle, start, abort, pause, and resume 
options. GUI has been developed for commanding and telemetry 
acquisition of encoder positions, limit switch positions ete. 


9 TESTING 


Testing to be done with over all zero g fixture In this method, 
the centre of gravity of the 2 arms and the wrist are found out 
individually using the CAD. In this case, the weight of the robotic 
manipulator may be offloaded by attaching counter balance at re- 
quired points using pulleys connected on guide rails of overhead 
zero-g setup or bungee cord system. All 6 DOF of the robotic arm 
may be demonstrated using this set up (Figure 16). 


Vision based Navigation, Guidance and Control af 6 DOF Robotic manipulator for Space Berthing application 


Figure 16: Overhead Zero g set up 


10 CONCLUSION 


In this paper, a method to compute 6 DOF freedom using color 
target is presented and test results are shown. The output was 
interfaced with inverse kinematics software to generate joint angle 
for robotic arm, Various aspects and design of robotic manipulator 
design, joint types were simulated and fabricated and tested in 
an integrated way. The method and test results are explained in 
this paper. This method can be used for On Orbit service of the 
spacecraft with suitable modification. 


11 FUTURE WORK 


For future version, itis proposed improve the vision based detection 
algorithm with some existing feature and detect similar or better 
accuracy. Avionics can be improved with Brushless DC (BLDC) 
motor based 6 DoF robotic arm, State space based BLDC motor 
control with Hall sensor/magnetic encoder in closed loop will be 
the implemented for the same. The prime challenge will be sizing 
of BLDC motor with gear train, 
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